There exist numerous applications in which real time data analysis may be required. For example, data events may be collected in a financial setting to identify potentially fraudulent activity, in a network setting to track network usage, in a business setting to identify business opportunities or problems, etc. Challenges however arise when analyzing data events in real time since historical data values are typically necessary to identify trends and patterns. Namely, accessing historical data can be a relatively slow process, and thus limits real time processing. There exist various known techniques (e.g., running estimates, etc.) for analyzing data events in real time (or near real time) when the events occur at regular intervals. However, it is much more difficult to analyze data events in real time when the events occur at irregular time periods.
One way to handle events occurring at irregular time periods is to collect a set of irregular (low level) events and generate regular (high level) events. For example, the low level event may be someone making an automated teller machine (ATM) withdrawal, taking a flight, logging on to a system, etc. The higher level event may be the total number of withdrawals in a day, the number of flights taken in a month, or the number of logins to a data access system in a shift. If more numeric information is available with the low level event, e.g., dollars withdrawn, flight miles or bytes transferred, these may then be summed for the high level event to give, e.g., total dollars withdrawn in a day, total flight miles in a month, or the total data transferred in a shift.
The generation of higher level events has drawbacks. Namely, the collection of event data is dependent on a higher level regular time interval that may be arbitrarily chosen. This time interval must be long enough to collect enough statistically significant lower level events. However, because analysis cannot take place until the end of the higher level event time interval, real time processing is again limited. For example, it would be best to observe irregular data transfer patterns as they happen, rather than at the end of the shift.
Accordingly, a need exists for a system and method of providing real time data analysis for irregularly occurring events.